AI Model Captures Human Reading for Personalised Text and AR
Researchers at Aalto University and international partner institutions have developed an artificial intelligence model of how humans read, describing it as the most accurate model yet. The model uses reinforcement learning, a type of AI also used in robotics, to explain and recreate the choices readers make as they move through text.
The study is due to be published on Monday, 10 August, in Nature Human Behaviour. The researchers said the model could support augmented reality displays and help tailor complex texts to different readers and everyday situations.
Earlier models were trained using large datasets that paired text snippets with eye-tracking data. Those systems could mimic aspects of human reading behaviour, but they lacked an understanding of the content and did not generalise well across languages or contexts.
The new model instead follows the psychological mechanisms readers use to direct their attention. It is designed to represent how understanding is built as the eyes move through words, sentences and paragraphs, capturing more than the visible pattern of where a reader looks.
The researchers said understanding how human memory supports reading is important for developing customisable apps, services and products. Written material has traditionally been produced for mass use rather than for a person in a specific situation, creating a role for systems that can adapt text to different users and circumstances. Rationality guides the model. Under this approach, readers continually decide where to look next to improve their understanding as much as possible within the time available. The decisions about where to direct attention are made at three levels: the word, the sentence and the text.
Those decisions are influenced by factors including a reader’s language, memory capacity, vision and eye speed. A fast reader with a good memory may move quickly from one paragraph to another, while a reader with a poorer memory may be more likely to return to earlier parts of the text.
The reading process also involves decisions about where to look, what to skip and when to backtrack. The model is designed to represent those choices as readers allocate their attention while processing text. The researchers added characteristics of individual readers as adjustable parameters. The model could then learn the best strategy for directing attention for different reader profiles.
The researchers trained it using AI-based reinforcement learning across millions of texts. The training was used to optimise the model’s eye movements so that it could understand what it reads rather than imitate patterns found in human eye-tracking data.
As the model reads, it forms a condensed description of the text’s content. When a crucial word or clause is missing from that description, the model can direct its gaze towards the information needed to improve its understanding.
Its understanding can then be tested by examining what it retains from the text within a given amount of time and under the specified constraints. This shows what information the model has taken from the material while it reads. The model’s attention-allocation decisions with real human eye-tracking data. They found that the model’s decisions mirrored the behaviour of human readers, producing a model of an average reader that can be tailored to different reader profiles.
The researchers said the model could support new reading tools and personalised text design. One example is smart glasses that could pace and lay out on-screen text according to the situation and the user’s needs, while also helping customise texts for different readers.
The same source text could also be adapted for different audiences. A convoluted piece of legal writing, for example, could be used to produce versions that are more comprehensible for different readers.
The next step for the research team is to evaluate how the model can be used to help individuals with dyslexia and people with low language proficiency, and to examine how the approach can support users in real-time situations.
One example given by the researchers involves drivers. Text could be designed to help drivers without distracting them. The research team said the new understanding of how people read could be used to explore further applications and support future uses.
The study involved researchers from Aalto University, The Hong Kong University of Science and Technology, City University of Hong Kong and the National University of Singapore.
